{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ab5dc2ff",
   "metadata": {},
   "source": [
    "# 2.5 自动求导 "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60a785cf",
   "metadata": {},
   "source": [
    "   问题1：为什么要引入自动求导\n",
    "   \n",
    "    · 在2.4节中已经强调了求导在深度学习中算法优化的关键步骤，虽然求导的计算很简单，但对于复杂的模型，手动进行更新慢且容易犯错\n",
    "    · 深度学习框架通过自动计算导数，即'自动求导'（automatic differentiation），来加速项目\n",
    "    · 在实际中，根据我们设计的模型，系统会构建一个’计算图‘（computational graph），来跟踪计算是哪些数据通过哪些操作组合起来产生输出\n",
    "    · 自动求导使系统随后反向传播梯度，这里的’反向传播‘（backpropagate）只是意味着跟踪整个计算图，填充关于每个参数的偏导数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b39523d",
   "metadata": {},
   "source": [
    "2.5.1 一个简单的例子\n",
    "\n",
    "    · 假设我们想对函数𝑦=2𝐱^⊤𝐱 关于列向量 𝐱 求导。首先，我们创建变量 x 并为其分配一个初始值"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a1349b3",
   "metadata": {},
   "source": [
    "    · 方法一"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6b290ab6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 1., 2., 3.])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch \n",
    "\n",
    "x = torch.arange(4.0)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8afada0c",
   "metadata": {},
   "source": [
    "    · 在我们计算 𝑦 关于 𝐱 的梯度之前，我们需要一个地方来存储梯度。我们不会再每次对一个参数求导时都分配新的内存。因为我们经常会成千上万次地更新相同的参数，每次都分配新的内存可能很快就会将内存耗尽\n",
    "    · 注意：标量函数关于向量 𝐱 的梯度是向量，并且与 𝐱 具有相同的形状"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c5ea4170",
   "metadata": {},
   "outputs": [],
   "source": [
    "x.requires_grad_(True) # equal to ‘x = torch.arange(4.0, requies_grad(True))\n",
    "x.grad # 默认值 是 None"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91074a86",
   "metadata": {},
   "source": [
    "    · 给 y 赋值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a4e95d35",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(28., grad_fn=<MulBackward0>)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = 2 * torch.dot(x, x)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b45e5ce3",
   "metadata": {},
   "source": [
    "   问题2：何为反向传播\n",
    "   \n",
    "    · x 是一个长度为 4 的向量，计算 x 和 x 的内积，得到了我们赋值给 y 的标量输出。接下来，我们可以通过调用’反向传播‘函数来自动计算y关于x 每个分量的梯度，并打印这些梯度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6d8e820b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0.,  4.,  8., 12.])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daa37bde",
   "metadata": {},
   "source": [
    "    · 函数 𝑦=2𝐱^⊤𝐱 关于 𝐱 的梯度应为 4𝐱"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d32c5853",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([True, True, True, True])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad == 4 * x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3e0b380",
   "metadata": {},
   "source": [
    "    · 现在让我们计算 x 的另一个函数\n",
    "    · 方法二"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "008da4f4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1., 1., 1., 1.])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 在默认情况下，PyTorch会积累梯度，我们需要清除之前的值\n",
    "x.grad.zero_()\n",
    "y = x.sum()\n",
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3eab881",
   "metadata": {},
   "source": [
    "2.5.2 非标量变量的反向传播\n",
    "\n",
    "    · 当 y 不是标量时，向量y关于向量x的导数的最自然解释是一个矩阵。对于高阶和高维的 y 和 x，求导的结果可以是一个高阶张量。\n",
    "    · 然而，虽然这些更奇特的对象确实出现在高级机器学习中（包括深度学习中），但当我们调用向量的反向计算时，我们通常会试图计算一批训练样本中每个组成部分的损失函数的导数。这里，我们的目的不是计算微分矩阵，而是批量中的每个样本单独计算的偏导数之和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d15d0172",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 2., 4., 6.])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 对非标量调用`backward`需要传入一个`gradient`参数，该参数指定微分函数关于`self`的梯度。\n",
    "# 在我们的例子中，我们只想求偏导数的和，所以传递一个1的梯度是合适的\n",
    "x.grad.zero_()\n",
    "y = x * x\n",
    "\n",
    "y.sum().backward() # 等价于y.backward(torch.ones(len(x)))\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4942558f",
   "metadata": {},
   "source": [
    "2.5.3 分离计算\n",
    "\n",
    "    · 有时，我们希望将某些计算移动到记录的计算图之外。例如，假设y是作为x的函数计算的，而z则是作为y和x的函数计算的。 \n",
    "    · 若我们想计算 z 关于 x 的梯度，但由于某种原因，我们希望将 y 视为一个常数，并且只考虑到 x 在y被计算后发挥的作用。\n",
    "    · 在这里，我们可以分离 y 来返回一个新变量 u，该变量与 y 具有相同的值，但丢弃计算图中如何计算 y 的任何信息。换句话说，梯度不会向后流经 u 到 x。因此，下面的反向传播函数计算 z = u * x 关于 x 的偏导数，同时将 u 作为常数处理，而不是z = x * x * x关于 x 的偏导数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e6dcb4d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([True, True, True, True])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad.zero_()\n",
    "y = x *x\n",
    "u = y.detach()\n",
    "z = u * x\n",
    "\n",
    "z.sum().backward()\n",
    "x.grad == u"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f2dc508",
   "metadata": {},
   "source": [
    "    · 由于记录了 y 的计算结果，我们可以随后在 y 上调用反向传播，得到 y = x * x 关于的x的导数，这里是 2 * x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e0b8b135",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([True, True, True, True])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad.zero_()\n",
    "y.sum().backward()\n",
    "x.grad == 2 * x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96577047",
   "metadata": {},
   "source": [
    "2.5.4 Python控制流的梯度计算\n",
    "\n",
    "    · 使用自动求导的一个好处是，即使构建函数的计算图需要通过 Python控制流（例如，条件、循环或任意函数调用），我们仍然可以计算得到的变量的梯度。\n",
    "    · 在下面的代码中，while 循环的迭代次数和 if 语句的结果都取决于输入 a 的值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ac579a62",
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(a):\n",
    "    b = a * 2\n",
    "    while b.norm() < 1000:\n",
    "        b = b * 2\n",
    "    if b.sum() > 0:\n",
    "        c = b\n",
    "    else:\n",
    "        c = 100 * b\n",
    "    return c"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99c6320a",
   "metadata": {},
   "source": [
    "    · 开始计算梯度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d525d743",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = torch.randn(size=(), requires_grad=True)\n",
    "d = f(a)\n",
    "d.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c13fedd9",
   "metadata": {},
   "source": [
    "    · 我们现在可以分析上面定义的 f 函数。请注意，它在其输入 a 中是分段线性的。\n",
    "    · 换言之，对于任何 a，存在某个常量标量 k，使得 f(a) = k * a，其中 k 的值取决于输入 a。因此，d / a 允许我们验证梯度是否正确"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "32e7940a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.grad == d / a"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7be032b6",
   "metadata": {},
   "source": [
    "2.5.5 小结\n",
    "\n",
    "    · 深度学习框架可以自动计算导数。为了使用它，我们首先将梯度附加到想要对其计算偏导数的变量上。然后我们记录目标值的计算，执行它的反向传播函数，并访问得到的梯度"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "591076e5",
   "metadata": {},
   "source": [
    "2.5.6 练习\n",
    "\n",
    "    · 1、为什么计算二阶导数比一阶导数的开销要更大？\n",
    "    · 2、在运行反向传播函数之后，立即再次运行它，看看会发生什么。\n",
    "    · 3、在控制流的例子中，我们计算 d 关于 a 的导数，如果我们将变量 a 更改为随机向量或矩阵，会发生什么？此时，计算结果 f(a) 不再是标量。结果会发生什么？我们如何分析这个结果？\n",
    "    · 4、重新设计一个求控制流梯度的例子。运行并分析结果。\n",
    "    · 5、使 𝑓(𝑥)=sin(𝑥) ，绘制 𝑓(𝑥)  和 𝑑𝑓(𝑥)𝑑𝑥 的图像，其中后者不使用 𝑓′(𝑥)=cos(𝑥)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef08fdfe",
   "metadata": {},
   "source": [
    "练习1：为什么计算二阶导数比一阶导数的开销要更大？"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5783717f",
   "metadata": {},
   "source": [
    "    · 比如有一个f(x) = 2*x^2 + x, 若对fx求一次偏导，则f’ = 4*x + 1；\n",
    "    · 再求一次偏导，系统就会调用f‘’，这样多次调用内存的开销就会增大"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "600db09a",
   "metadata": {},
   "source": [
    "练习2：在运行反向传播函数之后，立即再次运行它，看看会发生什么"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4af0fbe5",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-14-fb0d6192d3b2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0mz\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m \u001b[0mz\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[1;32m    243\u001b[0m                 \u001b[0mcreate_graph\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    244\u001b[0m                 inputs=inputs)\n\u001b[0;32m--> 245\u001b[0;31m         \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mautograd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    246\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    247\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mregister_hook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[1;32m    145\u001b[0m     Variable._execution_engine.run_backward(\n\u001b[1;32m    146\u001b[0m         \u001b[0mtensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgrad_tensors_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 147\u001b[0;31m         allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag\n\u001b[0m\u001b[1;32m    148\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    149\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time."
     ]
    }
   ],
   "source": [
    "# 采用分离计算为例\n",
    "x.grad.zero_()\n",
    "y = x *x\n",
    "u = y.detach()\n",
    "z = u * x\n",
    "\n",
    "z.sum().backward()\n",
    "z.sum().backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e29af7f",
   "metadata": {},
   "source": [
    "    · 反向传播算法再调用一次后，中间变量的值已经被释放了；如果想要再次调用，可以.backward()或autograd.grad()时指定retain graph=True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6c5d2de",
   "metadata": {},
   "source": [
    "练习3：在控制流的例子中，我们计算 d 关于 a 的导数，如果我们将变量 a 更改为随机向量或矩阵，会发生什么？此时，计算结果 f(a) 不再是标量。结果会发生什么？我们如何分析这个结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2c3975cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Python控制流的梯度计算\n",
    "def f(a):\n",
    "    b = a * 2\n",
    "    while b.norm() < 1000:\n",
    "        b = b * 2\n",
    "    if b.sum() > 0:\n",
    "        c = b\n",
    "    else:\n",
    "        c = 100 * b\n",
    "    return c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "9fb287ee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.4579, 0.2813, 0.5260],\n",
       "        [1.4836, 0.1968, 0.4502]], requires_grad=True)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = torch.randn(size=(2, 3), requires_grad=True)\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5e4e3168",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "grad can be implicitly created only for scalar outputs",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-22-1827a78a53b9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mz\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mz\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[1;32m    243\u001b[0m                 \u001b[0mcreate_graph\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    244\u001b[0m                 inputs=inputs)\n\u001b[0;32m--> 245\u001b[0;31m         \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mautograd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    246\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    247\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mregister_hook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[1;32m    139\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    140\u001b[0m     \u001b[0mgrad_tensors_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_tensor_or_tensors_to_tuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgrad_tensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtensors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 141\u001b[0;31m     \u001b[0mgrad_tensors_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_make_grads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgrad_tensors_\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    142\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mretain_graph\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    143\u001b[0m         \u001b[0mretain_graph\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36m_make_grads\u001b[0;34m(outputs, grads)\u001b[0m\n\u001b[1;32m     48\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrequires_grad\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     49\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnumel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 50\u001b[0;31m                     \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"grad can be implicitly created only for scalar outputs\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     51\u001b[0m                 \u001b[0mnew_grads\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mones_like\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmemory_format\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpreserve_format\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     52\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: grad can be implicitly created only for scalar outputs"
     ]
    }
   ],
   "source": [
    "z = f(a)\n",
    "z.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f59bca3",
   "metadata": {},
   "source": [
    "    · Grad只能隐式地为标量输出创建\n",
    "    · 我们发现z是个张量，但是根据要求output即z必须是个标量，当然张量也是可以的，就是需要改动一处代z.backward(torch.ones_like(a))\n",
    "    · 我们的返回值不是一个标量，所以需要输入一个大小相同的张量作为参数，这里我们用ones_like函数根据a生成一个张量。个人认为，因为要对a和y分别求导数，所以函数z必须是求得的一个值，即标量。然后开始对a求偏导数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "997cc8c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[ 468.9363,  288.0214,  538.6370],\n",
       "         [1519.2421,  201.4998,  461.0000]], grad_fn=<MulBackward0>),\n",
       " tensor([[4096., 4096., 4096.],\n",
       "         [4096., 4096., 4096.]]))"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = f(a)\n",
    "z.backward(torch.ones_like(a))\n",
    "z, a.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "914e3acf",
   "metadata": {},
   "source": [
    "练习4：重新设计一个求控制流梯度的例子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "700b9c41",
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(a):\n",
    "    b = a * 2\n",
    "    while b.norm() < 10000:\n",
    "        b = b * 2\n",
    "    if b.sum() > 0:\n",
    "        c = b\n",
    "    else:\n",
    "        c = 10 * b\n",
    "    return c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "93b2d5bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[40960., 40960., 40960.],\n",
       "        [40960., 40960., 40960.]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = torch.randn(size=(2, 3), requires_grad=True)\n",
    "d = f(a)\n",
    "d.backward(torch.ones_like(a))\n",
    "a.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e37702c",
   "metadata": {},
   "source": [
    "练习5：使 𝑓(𝑥)=sin(𝑥) ，绘制 𝑓(𝑥) 和 𝑑𝑓(𝑥)𝑑𝑥 的图像，其中后者不使用 𝑓′(𝑥)=cos(𝑥)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "8ebcc3d5",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABS5UlEQVR4nO2dd1xUx/r/30NHRKSooNhjT2zBrrFEY1eIii3WJOrV5KcmuZFovjdFTWKiMe0aS4wtVizYe48de1fsGFFERETq7vz+YOGiUpfdPXvgvF+vfbHnnDkzn9Xd88w8M/M8QkqJhoaGhkbhxUZpARoaGhoayqIZAg0NDY1CjmYINDQ0NAo5miHQ0NDQKORohkBDQ0OjkGOntABj8PLykhUqVFBahoaGhoaqOH78+EMpZYkXz6vSEFSoUIHQ0FClZWhoaGioCiHErczOa64hDQ0NjUKOZgg0NDQ0CjmaIdDQ0NAo5GiGQENDQ6OQoxkCDQ0NjUKOSQyBEOJPIcQDIcS5LK4LIcQvQogwIcQZIUT9DNcGCSGuGl6DTKFHQ0NDQyP3mGr56HzgN2BhFtc7AlUMr0bA70AjIYQH8AXgB0jguBBinZQy2lgher2e2+F3uf8gElskQhhbk4aUoENQqmQJyvmWwcZGG0BqaBRETGIIpJT7hBAVsinSHVgoU2NeHxZCFBdC+ACtgO1SykcAQojtQAdgqbFazly4zKnwGH4/9pjIZzr0WpRto7ERUKKILe/WjaPyrXBaNW2Ira2t0rKsjpiYGM6ePUvz5s0BOHDgANHR0XTq1EkVxvPhw4ccOHCAq1ev8uzZM9zc3KhWrRpNmjTBzc1NaXkaFsBSG8rKAHcyHIcbzmV1/iWEEMOAYQDlypXLsqHkhGd8sy+KJH0+FWugl3A/TsfUQ0/4K6AkC1dtYFDPrqp4uFmSCRMm8McffxAbG4u9vT1//fUXq1at4v79+wA8evQId3d3hBUNT6WU7Nixg2nTprFt2zYyy0tib29Ply5dGD9+PH5+fgqo1LAUqvlFSylnSyn9pJR+JUq8tEM6HRuBZgRMTJIebG1sOBh6irsRD5SWozjh4eH07NmTsLAwAEaPHs2WLVvSH/Tff/89u3fvRgiBXq+nZcuWDB48WEHFzxMWFsabb77JW2+9xenTp5kwYUL6KCY5OZnIyEh27tzJhx9+yN69e2nQoAE9e/ZMN2waBQ9LGYK7QNkMx76Gc1md17BCbG1siYl9qrQMxdHpdBw5coSzZ88CUKVKFVq1aoWdXeoA29XVlVq1agGpc1bvvvsuffv2VUxvRhYuXEjt2rU5fvw4//3vf7l58yYTJ06kadOmFC9eHDs7O7y8vGjTpg3Tpk3jxo0bfPXVV2zYsIHXXnuNzZs3K/0RNMyApQzBOmCgYfVQYyBGSnkP2Aq8JYRwF0K4A28ZzmlYJRK9vvAOtw4dOgRA+fLluXbtGgEBATneY2dnx5gxY+jQoQMAf/75J19//XWmrhhzotfrCQoKYtCgQTRu3JgLFy4wcuRIHB0ds72vWLFi/Oc//+H48eP4+PjQpUsXZs2aZSHVGpbCVMtHlwKHgGpCiHAhxLtCiBFCiBGGIpuA60AYMAcYCWCYJJ4IHDO8vk6bONbQsCZCQkJo2rQp69evB8DBwcGoeg4ePMiBAwfQ6XSmlJctUkpGjRrFlClTGD58OFu3bqVMmUyn4rKkVq1aHDx4kA4dOjBixAi++eYbM6nVUAJTrRrKdtxrWC00KotrfwJ/mkJHQSB673yentmB/tljPDuNweXVNtybPwa3JoG4VG+eqzriLuzlybEQvAf+aFUTlGqmS5cuzJw5k06dOuWrnjlz5pCQkICdnR0pKSnp7iRzIaVk7NixzJw5k08//ZTvvvvO6O+Ei4sLa9euZciQIUyYMAE3NzdGjcr0Z62hMlQzWVwYSPznMk8Or8Szwwf4jlpEkeoteHpmByApUq1ZruspUuMNZHIicRf2mE1rYWHr1q3ExMRgZ2fH8OHD8718VgiBs7MzCQkJdO3alRkzZphIaeb8/PPP/Pzzz4wZMyZfRiANOzs75s2bR7du3fjggw8IDg42kVINJdEMgRWRHP0PCBuKVGmMbVF3bOwdiQ0NwbVOhzz9gIUQFK3djtjQtWZUW/CJioqiR48efPbZZyav297eHmdnZ6NdTLlh8+bNfPzxxwQEBDBt2jSTjQ7t7OxYvnw5TZo0YciQIZw7l2lAAQ0VocrENHnl0Y7ZJD24bvF2HUpWwqPtsFyVfbhxOnHndgJwa0oXAHwG/0Lyw9s4V2mcXi4h/CL3lwZRovs4ilRtmnru1hnur/g/Svb4D86VXgfAuWoTonfPJTnqDvaeZdHIO56enmzZsoXq1aubvG5bW1tWrVplNtfd7du36devH6+99hqLFi0y+d4PJycnVq5cyeuvv46/vz+hoaEUL17cpG1oWA5tRGAleLQdhvub74OwwXfUInxHLSLhzllsi3pi5+qZXs7JtwbFm/UjavMvpDx5gO5ZDA83TKWYX/d0IwBgX9wbmyLFSbh9VomPo2qklFy9ehWA5s2b4+XlZZZ20ozAzp07GTRokMlWZOl0Ot555x1SUlJYtWoVLi4uJqn3RUqXLs3KlSu5efMmH3zwgVna0LAMhWJEkNteuZLYOLpg45j6g7Ut6g5AyuP72GYwAmkUa9KLhNtneLh+KsLBGVtXT4q/MfClcnaunqQ8jjCv8ALI4sWLGTJkCPv376dx48Y535BPwsLCOHbsGA8ePMDb2zvf9X377bfs37+fhQsXUrlyZRMozJpmzZrxn//8hy+++IKuXbvSu3dvs7anYR60EYEVI1OSEHYv+5CFsMGzy8ckRd4i8c55vLqNQ9i+bNOFrT0yJdESUgsUnTp14osvvqBBgwYWaW/YsGGcOHHCJEbg4sWLTJw4kd69ezNgwAATqMuZ8ePH07BhQ/71r38REaF1PNSIZgisGJsibujjYzO9lnz/OjI5AZmShO5JZKZldAlPsXHWgoblBSklHh4efP755xYLsCeEwMnJiaSkJBYsWGD0ZjO9Xs/w4cNxcXHhl19+MbHKrLGzs2PhwoXExcUxduxYi7WrYTo0Q2DFOJaqTMrje0hdynPndU+jebhpOm5NAnGt34WHG6ahe8Fg6JMTSHl8DwefKpaUrGo2btxIp06diIzM3LCam+DgYAYPHszu3buNun/u3Lns37+fqVOnUrJkSROry55q1aoxfvx4li1bxrZt2yzatkb+0QyBFeNYvjYAif9cSj8npeThxh+x9/DFrWkf3FsPwbaIG1Gbf37u3sTwiwhbe5zKvmpRzWomKiqK6OhoxUIv9+3bl71799KmTZs83xsZGcmnn35Kq1atGDJkiBnU5UxQUBBVq1Zl5MiRJCZqLkk1oRkCK8bWqShFqr9B3Pn/9RCfHFlFUsRVvLp+grCxRdja49XtUxJunSb2xIb0cnEXduNSsxU2Ds5KSFclAwcO5NChQ2Zd258dNjY2vPHGGwB5fpB+9dVXxMbGMmPGDMV2kzs6OvLLL79w7do1s2+U0zAtmiGwIoq+1pbyn6577pxb00CeXfqblNio1OPGPSk7ehl2xf4XitveowzlxgbjWj91/0HKk0jirx7BrUmg5cSrmHv37rF9+3YAqwjJsXnzZsqXL8/NmzdzVf7y5cvMmjWLYcOGUaNGDfOKy4H27dvTvn17Jk6cyKNHWtgwtaAZAivH3r00Hu0/ICUm97HgU2Lu49HhQ+zcLOsnVis///wznTp14u5d64iA/tprr9GsWbNcTxoHBQXh7OzMl19+aV5hueSHH34gJiaGSZMmKS1FI5dohkAFuNRogZNvzVyXdyr7aq4D1GmkulW2bduW54ic5sLX15dVq1ZRsWLFHMvu37+fkJAQgoKCLD5BnBWvvfYaQ4YM4bfffuPatWtKy9HIBZoh0CjUSClxdHSkdevWSkt5ifv37zN79uwsr0spmTBhAqVLl2bMmDGWE5YLJk6ciJ2dHRMnTlRaikYu0AyBRqHl5MmT1K1blwsXLigtJVPmz5/PyJEjuX498zhZu3fvZv/+/Xz22WcUKVLEwuqyx8fHhxEjRvDXX3+lp/TUsF40Q6BRaImJiaFIkSKULl1aaSmZMmrUKC5evEilSpUyvf7VV19RunRp3nvvPQsryx2ffvop9vb2TJ48WWkpGjlgqgxlHYQQl4UQYUKIoEyuTxdCnDK8rgghHme4pstwbd2L92pomItWrVpx6NAhq42aWbRoUapUSd0Q+GJAuj179rBv3z6CgoJwcnJSQl6OeHt7M2LECBYtWqTNFVg5+TYEQghb4L9AR6Am0FcI8dzMppRyrJSyrpSyLvArsDrD5fi0a1LKbvnVo6GRG/bv309ycrLSMnLFZ5999lJ+5C+//BIfHx/ef/99hVTlDm1UoA5MMSJoCIRJKa9LKZOAZUD3bMr3BZaaoF0NDaO4ffs2rVu3Vs3DycvLi/Lly6fnOT58+DB79+7l3//+t9WOBtLw8fHhvffe46+//rKa5bkaL2MKQ1AGuJPhONxw7iWEEOWBisCuDKedhBChQojDQgj/rBoRQgwzlAtVKhaMRsGgbNmyhISEMGyY9YcnB/j444/55Zdf0oPgTZs2jeLFi1vt3MCLjB07Fp1Ox6+//qq0FI0ssPRkcR9gpZRSl+FceSmlH9AP+EkIkWkAdSnlbCmln5TSr0SJEpkVKbA8/nsxd2f9zwXw5Ph6wv87iFtTuvL478Xp5x+s+pqYI6szqyJTEu9eJHzGEPTJhSsujBCCLl26WO0kcVacP3+eo0ePsnr1akaMGIGrq6vSknJFpUqV6NGjBzNnziQ2NvNouhrKYgpDcBfImAvR13AuM/rwgltISnnX8Pc6sAeoZwJNBZaU2Ciid87BrUkvyoxaQLGGbwMQf/MUifeuUOz1Lrmuy7FMDexLlOPJsTXmkmt1/PHHH/z4449Gh3pWigcPHlC3bl3+9a9/YWtry4cffqi0pDzx8ccfExMTw9y5c5WWopEJpjAEx4AqQoiKQggHUh/2L63+EUJUB9yBQxnOuQshHA3vvYBmgHUu6rYSUmIiQOpxfqURdkU90oPKxR4LoWitNpkmsskO19rteXpi40uhrgsqO3fuZNOmTVYRUygvlCxZkjlz5nDx4kX69eunutFMo0aNaNGiBdOnTyclpXB819REvg2BlDIF+ADYClwEVkgpzwshvhZCZFwF1AdYJp/vitUAQoUQp4HdwHdSSrMYgoglQTw9uyNVsy4l9dgQ1VOfnEDEkiDiLu5LPU6MI2JJEM8uHwRA9ywm9TjsSOrx02gilgQRf/04kBrkLWJJEPE3TwGQ/DiCiCUvraLNFTIliait/+X29EDu/NSbqK3/RepSV7c8/nsx9xePA+Du70O4NaULKTH30cU/If7GCZyrNEmvJzn6H25P78WTYyH/O/fwDrd/7EHsqS3p55wr+6FLiCXh1mmj9KqNpUuXsm6dOlcpR0REEB8fz0cffaS0FKP45JNPuH37NmvWFJ4RqFowyRyBlHKTlLKqlLKylHKy4dx/pJTrMpT5UkoZ9MJ9B6WUr0kp6xj+FvpxY/TeBTy7chCvLh/hPWAqNvZOxJ7YCECxhm9Twn88AD6Df8Z31CJsXb1IDE+1nY7e/5tesXcvjcdbI4neO5/EiDBkShKR66bgXKkBrnU7pJcTdg44lKxIwu0zFvyUyhAfHw9gdbtwc4NOp2PmzJnUr1+f2bNnq861BdC5c2cqVKighai2QgrNzmLvft9R9LW2AAhbu9TjWqnxZWzsnfDu9x0uNVJjwds4uuDd7zuKVGsKgG0Rt9TjVxqlHhd1x7vfdzhXeh0Au2IlUo8r1AXAvrg33v2+y7NGfVICsSc3UbzFAIpUaYy9Z1nc27yLvUfqIiwbB2dsnIumvnd2w7aoO8LGlpTH97FxLvaSW6horda41GjFw3XfE7V1BvqkeDw7vuxbtnX1KvBJ7o8ePYqPjw9///230lKMYvPmzdy6dYvGjRuzYsUKbt++rbSkPGNra8uIESPYs2eP1Yb1KKwUGkOgBlIe3wNdMo5lno8p71gm+8ijMiURYWef6TWPdiNAryPu/C5KdP0EG0eXl8oIWwdkSpLxwlWAq6sr3bt3p3bt2kpLMYoZM2bg4+PD999/z507dyhfvrzSkozi3XffxdHRkd9//11pKRoZ0AxBAcCmiBv6hKeZXkt5/A8pT1MThCRH38u0jD4htsAnua9RowYLFiygWLFiSkvJM9euXWPLli0MGzYMFxcXHB0dkVKqZmd0Rry8vAgMDGTBggXaUlIrQjMEVoRdcR+wtSPx7sXnzr94/CIOpSojk+JJefLgufP6pAQi136PS403cG/9Lo+2/05y9D8v3Z8ceRNHn1fy/wGslEOHDhEeHq60DKOZNWsWNjY26eEkEhMTady4sWpDPI8cOZLY2FgWL16cc2ENi6AZAivCxsEJ17odebz/L55dPUJyVDjRu/8k+VH2DzGHUpWwLepBwu1zz51/tGMWSD0e7Ubg6tcNp7Kv8nDdD88tFU1+dBfd02icKvmZ5TMpjZSSoUOH0rdvX6WlGEV8fDxz587F398/PXGOo6MjTZs2pWrVqgqrM45GjRpRr149ZsyYocpJ74KIZgisjOItB1OkSmMebpjGvYUfoU+Mw7V+52zvEcKGonU6PJfkPu7ifuIu7Mar26fYODgjhMCz0xh0T6N4vG/h/8qd34NThbrYF/c222dSEiEEGzdu5KefflJailEEBwfz6NEjRo4c+dz56dOn88477yikKn8IIRg5ciRnz57lwIEDSsvRAIQaLbKfn58MDQ3N9Nrx48fpEVywV8Bkhi7hKf/MGU6pwK9xKJVplI6X0CfFc3f2+5QM+BzHMtWzLbuqlzdzgjcxclBvatdQZ09UjbRs2ZJ79+5x+fLllzbBJScnc+zYMZo2baqQOuOJi4vD29ubXr168eeffyotp9AghDhuCOnzHNqIoIBg61QUry4fp08M54aUmPsUbzEgRyOgVqKiohg2bJhqM2RdvXqVffv2MWTIkEx3Qn/77be0aNFClfMfLi4u9OnThxUrVmiTxlaAZggKEM4V61OkcoNcl3coUQHXOu3NqEhZTpw4weLFi4mLi1NailHMnz8fGxsbBg4cmOn1wYMHs3btWry91enWGzp0KHFxcQQHBystpdCjGQKNAku7du148OABderUUVpKntHpdCxYsIAOHTqkTxK/SLly5ejSpQt2dnYWVmcaGjduTPXq1TXXkBVQ4AyBlGCnrnhiVo+dAL3K5pLSUju6uLy8gU4NbN++nbt37zJ06NBsy8XFxTF9+nSOHTtmIWWmQwjB0KFDOXDgAJcvX1ZaTqGmwBkCnbClskfmu2w1jKOyhz2x8eravPTxxx/TuXNn1S5P/PPPP/Hy8qJr167ZlhNC8OWXX7JhwwYLKTMtAwYMwNbWlnnz5iktpVBT4AzBKxXKMqG5O9U87bWRQT6xE1DN056gpm5cv30XBOlZsqydChUqUL16ddWFm4bUSe61a9fyzjvv4OCQfVjxIkWKcPnyZb766isLqTMt3t7edOrUiQULFmjhqRVEnc7FbPD09MTdLZJPGiZS3MUJGxU+CKwFvZTExidz7eYd7tyPRK+TuLupI0TD6NGjlZZgNEuWLCEpKYkhQ4bkqrxaJ4vTGDp0KOvXr2fLli106ZL7xEoapqPAGQKAWtWrcu7KDZat34FbsaKqmEzbvXs3586cYdjw4ThaU0JyKUlOSSEm9ik9OrXFp6SX0opy5NKlS1SrVk2VowGARYsWUbdu3TwFyJsyZQonT55k2bJlZlRmHjp37oynpyd//fWXZggUwvqfkEZgY2ND727tKeHpwaVrN3hmiENvzTRr3JBTx48RfvsWjRo1VFrOc7gWdaF29ao09atj9Q/X6Oho6tSpQ1BQkCrdJVevXuXYsWNMnTo1T/el/b/odDrVuO/SsLe3JzAwkPnz5xMbG6uaXMwFCillvl9AB+AyEAYEZXJ9MBAJnDK83stwbRBw1fAalJv2Xn/9dVnQ0Ov1smbNmrJFixZKS1E1z549kwsXLpQXLlxQWopRfPHFF1IIIcPDw5WWYlH+/vtvCcgFCxYoLaVAA4TKTJ6p+Q4xIYSwBa4A7YBwUnMY95UZUk4KIQYDflLKD1641wMIBfwACRwHXpdSRmfXZnYhJtTM5MmT+fzzz7l16xblypVTWo6GhZFSUrVqVcqVK8fOnTuNqiMqKgpPT08TKzM/UkoqVapE1apV2bp1q9JyCizmDDHREAiTUl6XUiYBy4Duuby3PbBdSvnI8PDfTuroolCSFiFTjX5ea+DWrVssXLiQZ8+eKS3FKI4dO0ZYWBj9+/c36v5169ZRqlQpzp07l3NhK0MIQf/+/dmxYwcREYUvVpjSmMIQlAHuZDgON5x7kR5CiDNCiJVCiLJ5vBchxDAhRKgQIjQyMtIEsq2PSpUq0bhxY5YsWaK0FFWyYsUKBg8ezKNHuY+3ZE0sXrwYR0dHevToYdT9TZo04d///jfFixc3rTAL0b9/f/R6vdYRUgBL7SNYD1SQUtYmtde/IK8VSClnSyn9pJR+JUqUMLlAa6Ffv36cPn2a8+fPKy1FdXzyySecPXsWX19fpaXkmZSUFJYtW0aXLl1wczMuW1yJEiX49ttvVfn5ITWLXL169bSENQpgCkNwFyib4djXcC4dKWWUlDLRcPgH8Hpu7y1sBAYGYmNjw9KlS5WWojqEENSqVUtpGUaxc+dOHjx4YLRbKA0pJaGhoaoN2dC/f39CQ0O5cuWK0lIKFaYwBMeAKkKIikIIB6APsC5jASGET4bDbkBa7sWtwFtCCHchhDvwluFcoaVUqVK0bduWJUuWqDY8ghL8/PPPqk3dCKluoeLFi9OpU6d81RMfH0/r1q2ZNm2aiZRZlj59+iCE0EYFFibfhkBKmQJ8QOoD/CKwQkp5XgjxtRCim6HY/xNCnBdCnAb+H6nLSZFSPgImkmpMjgFfG84Vavr168eNGzc4cuSI0lJUw6lTpzh69KjSMozi2bNnrFmzhp49e+Lo6JivuooUKcL69euZMmWKidRZljJlytCmTRsWL16sdYQsSWZrSq39VRD3EWQkJiZGOjo6yg8//FBpKaoiOTlZaQlGsWLFCgnIXbt2KS3FKpg7d64EZGhoqNJSChxksY+gwAWdKwgUK1aMrl27snz5ci0QVy5ICzmthlAimbFixQpKlSrFG2+8YbI6Q0JC+O2330xWnyXx9/fHzs6OFStWKC2l0KAZAiulb9++PHjwgL179yotxarR6XTUqFFDtcnp4+Li2LhxIz169DBpaIiQkBB+//13VbpXPDw8aNu2LcHBwarUr0Y0Q2CldOzYERcXFy2NXw48ffqUVq1aUalSJaWlGMXGjRuJj4+nV69eJq33559/5uzZs1YfGyorAgMDuXHjBsePH1daSqFAMwRWirOzM126dGH16tWaeygb3NzcmDVrFt26dcu5sBUSHBxMqVKlaNGihUnrdXNzw8ZGvT/v7t27Y2dnp3WELIR6vymFgF69ehEZGcn+/fuVlmKVJCcnc/36daVlGI253EJprF+/nmbNmpGYmJhzYSvDw8ODdu3asWLFCs09ZAE0Q2DFdOzYkSJFimi9oizYu3cvlStXNjpAm9KkuYUCAwPNUr+DgwO2traqjd3Tq1cvbt68qbmHLIBmCKyYIkWK0LlzZ1avXo1Op1NajtVRs2ZNpk2bRpMmTZSWYhRpq4WaN29ulvrbt2/Pvn37KF++vFnqNzf+/v7Y29trq4csgGYIrJxevXpx//59zT2UCaVLl+ajjz6iSJEiSkvJM0+fPmXjxo307NnT7IlkEhMTSU5ONmsb5sDd3V1bPWQhNENg5XTq1AlnZ2dWrlyptBSr4tq1a2zbtk2VDzhIdQslJCSYfLXQi5w7d46SJUuyefNms7ZjLgIDA7l58yYFMf+INaEZAivHxcWFTp06sWrVKs09lIE///yTTp068fTpU6WlGMWKFSvw9vY2m1sojWrVqtG/f3/Kli2bc2ErpHv37tjb22vzZGZGMwQqoFevXkRERHDgwAGlpVgNn3/+OXv37sXd3V1pKXnm6dOnbNq0yWyrhTJib2/PjBkzqFevnlnbMRfu7u7a6iELoBkCFdC5c2ecnJy0XlEGnJ2dadasmdIyjGLDhg0kJCSYbbVQZty+fZtr165ZrD1T0qtXL27duqW5h8yIZghUQNGiRdPdQ2lxdQozy5cv57///a9qe4hr1qyhZMmSFjNkKSkp1KlTh8mTJ1ukPVOTtrls9erVSkspsGiGQCX07NmTe/fucfDgQaWlKE5ISAgLFixQZfiExMRENm3aRPfu3c3uFkrDzs6OhQsXMn78eIu0Z2rc3d1p1aoVq1evVq3xt3YKnSGIiYlRWoJRdOnSBUdHR61XBCxdupTt27crLcModu3axdOnT/H397dou127duWVV16xaJumJCAggCtXrnDx4sWcC2vkGZMYAiFEByHEZSFEmBAiKJPrHwkhLhiS1+8UQpTPcE0nhDhleK178V5TMnnyZMqVK6fKLfeurq60bduWNWvWaL0iMDqvr9KEhIRQtGhR2rRpY/G29+3bp9p5pjTDuWbNGmWFFFDybQiEELbAf4GOQE2grxCi5gvFTgJ+MjV5/Urg+wzX4qWUdQ0vs0YOa926NUFBQSQlJZmzGbMREBDAzZs3OX36tNJSFGPIkCGqzb6l1+tZu3YtHTt2xMnJyeLt//TTT0yYMEGVHYnSpUvTuHHjQm0IoqOjefLkiVnqNsWIoCEQJqW8LqVMApYB3TMWkFLullI+MxweJjVJvcVp2rQpn332Ga6urko0n2+6deuGjY1Nof0xSCl5+vQp8fHxSksxiiNHjnD//n2Lu4XS+OWXXzh16pQq51YgtSN0/Phxbt26pbQURShevDiRkZFm2URpCkNQBriT4TjccC4r3gUybnN0EkKECiEOCyH8s7pJCDHMUC40MjLSaLEpKSns3btXlZuzSpQoQbNmzQqtIRBCEBwczJdffqm0FKMICQnBzs4u3wnqjcXX11eV4TjSCAgIAFL/HQsjQggqV66Mvb29yeu26GSxEOIdwA/4IcPp8lJKP6Af8JMQonJm90opZ0sp/aSUfiVKlDBaw+rVq2nVqhWHDx82ug4lCQgI4OzZs6pdE54f1Di3k4aUkjVr1tC6dWuKFy+umI6NGzcyZMgQVbqHqlSpwquvvlooO0IXL15k7Nix3L171yz1m8IQ3AUy7l/3NZx7DiFEW2AC0E1Kmf6LllLeNfy9DuwBzLoFskOHDqxcuZK6deuasxmzUVh7RfHx8fj4+PDrr78qLcUoLl26xNWrVxVzC6Vx584dDhw4QFRUlKI6jCUgIID9+/eTH6+AGjl58iSzZs0y25JjUxiCY0AVIURFIYQD0Ad4bvWPEKIeMItUI/Agw3l3IYSj4b0X0Ay4YAJNWVKsWDF69OiBi4uLOZsxGxUqVKBu3bqFrlcUHx/Pu+++q9pQCWmGW+lMau+//z6XL1/Gy8tLUR3GEhAQgF6vZ906sy4wtDr69etHVFQU3t7eZqk/34ZASpkCfABsBS4CK6SU54UQXwsh0r71PwBFgeAXlonWAEKFEKeB3cB3UkqzGgKAJ0+eMHv2bK5evWrupsxCQEAABw8e5P79+0pLsRgeHh788MMPZg/SZi5CQkJo0KABvr6KrJNIx9bWFiGEKl1DAHXr1qVChQqFriMEqWFVzIVJ5giklJuklFWllJWllJMN5/4jpVxneN9WSlnqxWWiUsqDUsrXpJR1DH/nmkJPTsTHxzNixAjWr19vieZMTkBAAFJK1q5dq7QUi6DT6Thz5oxqH153797l6NGjiruF0tizZw+VKlXixo0bSkvJM0IIAgIC2L59O7GxsUrLsQhz5syhc+fOxMXFma2NQrezGKBUqVJcuXKFsWPHKi3FKF599VUqV65caHpFBw4coE6dOqo13GluDGsxBOXKlaNWrVqqDeEdEBBAUlISmzZtUlqKRZBSotfrzerOFmrsZfn5+cnCHonwk08+4ZdffiEyMlK1u2xzS1RUFKtWraJv376q3APSvn17bty4weXLl1W7ht+a0Ol0+Pj40KZNG5YtW6a0HFUhhDhuWKX5HIVyRACp+wk++eQTFi1apLQUowgICCA5OblQ9Io8PT0ZNmyYKo3A48eP2bVrFwEBAVZnBGJjY83qbjAXtra2+Pv7p2d5K8jExcVZxCVaaA2BnZ0de/bs4fz580pLMYomTZpQqlSpAr+M9Nq1a6xatUq1u4k3b95MSkqK1biF0rh16xZeXl4sWbJEaSlGERAQwNOnT9m5c6fSUszK4MGDadWqldnbsTN7C1bMkSNHLBYK2NTY2NjQvXt3lixZQkJCgiKxayzB0qVL+eKLL7h3755ZV02Yi5CQEEqVKkWjRo2UlvIc5cqVY/z48TRs2FBpKUbRpk0bXF1dWbNmDZ07d1Zajtno3LmzReZytDkCUidjrG3Ynhu2bNlCx44d2bBhQ4H9MaSkpHDmzBnq16+vtJQ8k5iYiJeXF/369WPWrFlKyylw9O3bl507d3Lv3j3VdugsjTZHkAUjR46kd+/eSsswijZt2lCsWLECvXrIzs5OlUYAlMs9kFuklJw/f1617tGAgAAiIyMLbLKmU6dO8ezZs5wLmoBCbwjKli1LxYoVlZZhFA4ODnTu3Jl169apMoheTixfvpxvv/1Wtek5lcw9kBv0ej1t2rRh0qRJSksxig4dOuDg4FAg58l0Oh3t27fn/ffft0h7mmtI5axYsYLevXuzf/9+1e66zYoRI0Zw5MgRTp48qbSUPKPX6yldujQtW7Zk+fLlSsvJkr1791K1alV8fHyUlmIUnTt35tKlS4SFhanSvZsVer2ePXv2UKxYMfz8XvLkGI3mGsoBtYZrKMi9opkzZ6p22K907oHc0rJlS9UaAUjdpHf9+nXOnTuntBSTYmNjQ5s2bUxqBLJtzyKtWDlBQUFUq1bNLAkfzE2xYsV48803CQkJUW0IhuxQ40ohSHUL2dvbK5Z7IC9s3ryZ+fPnKy3DKLp27YoQokDNk0kpmTFjhkVDgGiGgNSIkJMmTSIlJUVpKUbh7+/PtWvXVDvplxmDBg1iwoQJSsswioy5B9Sw63vevHl8//33quxIeHt706RJkwI1Ir5x4wajRo1i8+bNORc2EZohIDWF5QcffKDa3me3bt0QQhSYH4OUEnt7e7NkYrIE1pJ7ILf89ttvnD59WrU+9oCAAE6ePFlgUlhWqlSJW7du0a9fP4u1qRkCA/Hx8WzatEm1vaLGjRsXGEMghOCPP/5QdUpKUD73QG4pWbKkao0uQPfuqSnSC8r3H1I3/Fkyk51mCAwEBwfTuXNnjh8/rrQUo/D39+f48ePcvn1baSn5Rq1RMdMICQmhYcOGlCmTXepu62LVqlUEBgaqsiNUpUoVatWqVSAMwb1793jvvfe4cuWKRdvVDIGBbt26sW3bNmrXrq20FKNIc0OoPUdBUlISZcuW5ZtvvlFailFYW+6B3PLw4UOuXbvG48ePlZZiFAEBAezbt0+1KTjTOH/+PCtWrLD4whWTGAIhRAchxGUhRJgQIiiT645CiOWG60eEEBUyXPvMcP6yEKK9KfQYQ/HixWnXrh0ODg5KScgXVatWpUaNGqrvFSUmJjJ27FhatGihtBSjsLbcA7ll2LBhHD9+HHd3d6WlGIW/vz96vV61OSvSaNu2LQ8fPqRmzZoWbTffhkAIYQv8F+gI1AT6CiFe/BTvAtFSyleA6cAUw701Sc1xXAvoAMww1KcIDx8+ZNq0ady8eVMpCfnC39+fvXv38ujRI6WlGI2rqyv/+c9/VGsIQkJCqFq1KtWrV1daSp5ImyhW6y7u+vXr4+vrq/qOEKRGDLD0xL0pRgQNgTAp5XUpZRKwDOj+QpnuwALD+5XAmyL1k3YHlkkpE6WUN4AwQ32KEBcXxyeffMKuXbuUkpAv/P390el0bNy4UWkpRqHX6zl06JBqw2Wk5R7w9/dX5QqcLVu24OPjo8p5JiEE/v7+bNu2zWLxeUxNcHAwb7zxBhERERZv2xSGoAxwJ8NxuOFcpmUMye5jAM9c3guAEGKYECJUCBEaGRlpAtkvU758ee7cucPQoUPNUr+58fPzo0yZMqrtFR09epSmTZsSHBystBSjsNbcA7mlcuXKtGnTRrXJXgICAoiPj2fbtm1KSzEKGxsb7OzsKFGihOXbtniLRiKlnC2l9JNS+pnzH8rX19dsdZubtBwFW7ZsUWUil1q1arF06VI6dOigtBSjsNbcA7mlSpUqLF26lKpVqyotxShatGiBu7u7ancZ9+jRg127dikSUtsUhuAuUDbDsa/hXKZlhBB2gBsQlct7LcqzZ88YPny4VQcKyw5/f3+ePXvGjh07lJaSZ1xdXenTp49F10+bisTERDZt2kT37t2xsVFN/ypTIiMjVZnC0t7eni5durB+/XrVRQmIjY1VdH7GFN/YY0AVIURFIYQDqZO/614osw4YZHjfE9glUxcsrwP6GFYVVQSqAEdNoMlonJ2dOXz4sGonjFu2bImbm5vq3EM3btxg3rx5xMbGKi3FKKw990BuuXjxIt7e3qxcuVJpKUYREBBAdHQ0+/fvV1pKnkiLd6aUMch3qkopZYoQ4gNgK2AL/CmlPC+E+BoIlVKuA+YCi4QQYcAjUo0FhnIrgAtACjBKSqnoTKEQglOnTqlysg9ezlGglsxNISEhfPTRR7Rv316VSeqtPfdAbqlevTrffPMNTZs2VVqKUbz11ls4OTmlx3pSC+3bt6dSpUqKjSa1fATZoNYUlsHBwQQGBrJv3z7VLMOUUnLp0iVq1KihtJQ8o5bcA4WF7t27p8ceUuPv15xo+QjySK9evRgyZIjSMoxCjTkKhBCqNAKgntwDuUWv13PkyBHVxvj39/fnzp07qklodOrUKcX3/miGIAteffVV1T6YXF1dadu2rWpyFCxbtozPPvtMlfkgQF25B3KDTqejY8eO/PDDD0pLMYquXbtiY2Ojmo5Q//796dWrl6IaNNdQAWXOnDkMGzaMM2fO8NprryktJ1s+/fRTtm7dyunTp5WWkmeklFSrVo2KFSuydetWpeWYjIMHD1KjRg3Vhpxo1aoVUVFRnD17VmkpOXLs2DGSk5MtMi+juYaMQErJnTt3ci5ohaRlblJDr+j7779HrYZdbbkHckvTpk1VawQg1T107tw5wsLClJaSIw0aNFB8cl4zBNnw7rvv0qRJE1W4V15EbZmb1BoPX225B/LC8uXLmTt3rtIyjEIt0XhnzZplFXMxmiHIhgEDBvDNN9+oNvaNv78/J06csOrYMQMHDuSDDz5QWobRrFmzhkaNGqkq90BuWbp0KX/88YfSMoyiQoUK1K1b16p3GT9+/JgPP/yQVatWKS1FMwTZ0bp1awYOHIidXb63WyiCGnpFJUuWxMvLS2kZRhEeHs6xY8fSM2QVNObNm8eBAweUlmE0/v7+HDx4kPv37ystJVOKFy/OP//8w6hRo5SWohmCnHj8+LFVP0izo0qVKtSsWdOq3UNTp05VfUrKgIAAZYWYCXd3d1WHy/D390dKadU5Cry8vKyiI6Te/2ULsWDBAvz9/VUx6ZQZ1pyjwBo15YWQkBCqV6+uutwDeWH+/Pm8/fbbSsswitq1a1OxYkWrdA89ffqUvn37Ws0iCc0Q5EDfvn05dOgQlSpVUlqKUaTlKNiwYYPSUp4jJSWFKlWqEBT0UkI7VfDo0SP27NlTYEcDaTx79ozo6GhV5pFOy1GwY8cOq4thdfXqVXbv3m01/66aIciBkiVL0rhxY9UOkV9//XWrzFGQnJzM//3f/6l2E9aGDRvQ6XQF3hCMHDmS3bt3U7RoUaWlGIW/vz9JSUls2bJFaSnPUa9ePf755x/eeOMNpaUAmiHIFXfv3uXLL7+02kmn7MiYo8CaMjc5OzszZswYq/kh5JU1a9bg6+uLn99Le3MKJGrd9d2sWTO8vLysqiOUthzdxsbGajqY1qHCynn48CFff/01hw8fVlqKUfj7+xMfH281OQr0ej3bt28nKSlJaSlG8ezZM7Zu3aralJR5ZdWqVXh5eXHv3j2lpeQZW1tbunXrxsaNG63m+7Zjxw7q1q3LlStXlJaSjmYIckHt2rW5f/++apcJWluOgtDQUN566y3VRurcunUr8fHxBd4tlEbNmjXp06eP1TxI84q/vz8xMTHs2bNHaSlA6tyFp6cnZcuWzbmwhdBiDRUS+vfvz9atW4mIiFB8X0RCQgI7d+6kWbNmqsxGNnDgQDZs2MCDBw8U/7fUyJn4+HhKlCjBwIEDmTFjhtJyFEWLNZRPoqKi6Nevn1WvSc4Of39/oqKiOHjwoNJScHJyonPnzqo0AsnJyaxfv56uXbsWOiNw8+ZNVaawdHZ2pkOHDoSEhCiaDhJS9yVZ48gqX4ZACOEhhNguhLhq+PtSlCohRF0hxCEhxHkhxBkhRO8M1+YLIW4IIU4ZXnXzo8ecuLm5cfr0aVVOGIP15CgICwvjl19+ITo6WlEdxrJv3z4eP35caNxCaZw4cYKKFSuquiN07949jh07pqiOSZMmUbZsWaszBvkdEQQBO6WUVYCdhuMXeQYMlFLWAjoAPwkhime4/m8pZV3D61Q+9ZgNOzs7zp07x3vvvae0FKOwlhwFW7duZcyYMSQkJCimIT+sWbMGZ2dn3nrrLaWlWJS6devy888/06xZM6WlGEXnzp2xs7NTvCPUuXNnxo0bh4ODg6I6XiRfcwRCiMtAKynlPSGED7BHSlkth3tOAz2llFeFEPOBDVLKPGXKVnqOQE25gDOSlqPg9OnT1K5dWzEdt2/fply5coq1byx6vZ5y5crRsGFDVq9erbQcjTzSrl07wsPDuXjxotJSFMNccwSlpJRpa8oigFI5iGgIOADXMpyebHAZTRdCOGZz7zAhRKgQIjQyMjKfso1DSknLli0ZM2aMIu3nF2vJUaBGIwCpq53u3r1b6NxCaaSkpLBt2zbOnDmjtBSj8Pf359KlS1y6dEmR9o8cOUJ4eLgibedEjoZACLFDCHEuk9dzayll6tAiy+GFYcSwCBgipUybsfkMqA40ADyAcVndL6WcLaX0k1L6lShRIudPZgaEEDRv3lzR3nR+UDpHwfz58/nwww9VuzlpzZo12NnZ0aVLF6WlKIJOp6Nnz56qXXmTljNCqe//e++9R//+/RVpOydyXPYgpWyb1TUhxH0hhE8G19CDLMoVAzYCE6SU6buyMowmEoUQ84BP8qReASZPnqy0hHzh7+/Pp59+yq1btyhfvrxF275+/TqhoaGqTUKzZs0aWrVqperMXfnB0dGRXbt2UatWLaWlGEXZsmXx8/MjJCREkRhXwcHBVhfzKI38uobWAYMM7wcBL8VrFkI4AGuAhS/OBRiMByJ1e6Y/oHyqnlyQnJzMzZs3lZZhFErmKPj666+tYvmqMVy8eJHLly8XWrdQGn5+fjg7Oystw2gCAgI4cuQId+/etXjb1atXp0GDBhZvNzfk1xB8B7QTQlwF2hqOEUL4CSHSUhsFAm8AgzNZJrpYCHEWOAt4AZPyqcci9OjRQ7XuAaVyFKSt31ZrSIa0yWG17i43JXPmzGHOnDlKyzCKtI7QunXrLNru1KlTrSbkdGZoO4uNYPv27cTFxdG9e3dVPtgmTJjAlClTuH//Pp6enhZps1u3bpQqVUq1D5C6devi4uKi6oxdpqJjx44AbN68WWEleUdKSbVq1ahYsSJbt261SJuPHz+mdOnSfP7554wfP94ibWaFtrPYhLRr107VAccCAgLQ6XQW7RXVqVOHGjVqWKw9U3L16lVOnz5Nr169lJZiFaxcuVKVRgBSR6QBAQHs2rWLx48fW6TN4sWLExkZaRUpKbNCMwRGcv/+fZYsWaK0DKN4/fXXqVChAsHBwRZrc+LEiXz00UcWa8+UpP079ezZU2El1oGLi4vSEvKFv78/KSkpbNq0yWJturi44ObmZrH28opmCIxk6dKl9O/fn+vXrystJc8IIejVqxfbt2+3SLrIa9euKbqbOb+sXLmSJk2a4Ovrq7QUq+HXX39V7XxJo0aNKFWqlEXmye7fv0+HDh04fvy42dvKD5ohMJJ33nmHc+fOqTaFZWBgICkpKWZfPfT48WNq1KjBpEmqWAfwEteuXePkyZPaaCALEhMTlZaQZ9KSNW3atIn4+HiztnXjxg2uXr2Kk5OTWdvJL5ohMBIvLy/VrqeG/7mHVqxYYdZ27O3t+f3331WbAF1zC2XOhx9+yNq1a3F0zDIYgFXTs2dP4uLizJ7CsnHjxoSFhVn9s0IzBPkgPDycsWPHEhYWprSUPCOEIDAwkB07dpjVPeTi4sK7775r9T+ErAgODqZRo0aqDYthbszdozYXrVu3pkSJEmZNjpScnIyUUhWLSjRDkA+klMycOZMTJ04oLcUo0txD5vKVPn78mOXLl6syhj2k7oQ+ceKEtlooCxYuXIinpycPHmQaUMCqsbOzo0ePHqxfv95s38/Zs2fzyiuvEBUVZZb6TYlmCPJB2bJliYqKIjAwUGkpRlG/fn0qVqxoNvfQ+vXr6dOnDxcuXDBL/eZm5crUjfCaWyhz/Pz8+Ne//oVOp1NailH07t2bZ8+esXHjRrPUX6lSJd58802L7dXJD9qGskJOUFAQ06ZNIyIiwuRfWJ1Ox6FDh2jWrJkqhscv0qBBA4QQHD16VGkpGmZAp9Ph6+tL06ZNWbVqldJyLIK2ocxMJCcn07VrV6ZOnaq0FKPo1auX2dxDtra2NG/eXJVG4MaNG4SGhmpuoRyQUnLmzBkePnyotJQ8Y2trS8+ePdm0aZPJg8GFhYXx9OlTk9ZpTjRDkE/s7e1xdna2uoxDuaV+/fpUqlTJ5JvL1q1bx+TJk1W5vBA0t1BuuX79OnXq1OGvv/5SWopR9O7dm4SEBJOn4BwyZAht2rQxaZ3mRHMNaRAUFMTUqVNNGnvo448/Zu3atVy9elWVIwI/Pz+EEIrnuFUDS5cu5c0336RkyZJKS8kzaVnnXn/9dZPuqdm/fz9xcXF06NDBZHWaAs01ZGaklKocHkPq6iGdTseaNWtMVue0adM4c+aMKo3A5cuXOX78uNUmEbE2+vbtq0ojAKmbywIDA9myZYtJYw+1aNHC6oxAdmiGwEQMGjSIFi1aqDKUQr169XjllVdYunSpSestUqSISeuzFEuXLkUIQe/evZWWohpCQkIsHtrZVPTu3ZukpCSTjQjmzp3LtWvXci5oRWiGwEQEBgYyZsyY9Lj7akIIQb9+/di9e7dJEnb4+/vzxRdfmECZ5ZFSsnjxYtq0aYOPj4/SclTDlClT+Pnnn5WWYRQNGzakQoUKJtlcdv/+fYYNG2b2HfumJl+GQAjhIYTYLoS4avibaQ4/IYQuQ1KadRnOVxRCHBFChAkhlhuymamSLl26MHz4cGxtbZWWYhT9+/dHSsmyZcvyVY9Op8PT05NixYqZSJllCQ0NJSwsjH79+iktRVUEBwdbLL6/qUnbZb99+/Z8b/4qVaoUt27dYtiwYSZSZxnyOyIIAnZKKasAOw3HmREvpaxreHXLcH4KMF1K+QoQDbybTz2KEh8fz5o1a1S5waZq1ao0aNCAxYsX56seW1tb5s6dy8cff2wiZZZlyZIlODg4qDY2klL4+vpiZ5djCnSrpXfv3qSkpJhkP4Gvr68qNpFlJL+GoDuwwPB+Aal5h3OFIU9xGyAtj3Ge7rdG1q1bx9tvv83+/fuVlmIU/fv35+TJk1y8eNHoOv755x8TKrIsOp2OZcuW0blzZ4oXL660HNWxcuVK3n77bdXOk9WoUYNFixYZXcepU6cYOHAgd+7cMaEyy5BfQ1BKSnnP8D4CKJVFOSchRKgQ4rAQwt9wzhN4LKVMMRyHA2WyakgIMcxQR2hkZGQ+ZZuHrl27sn37dlq0aKG0FKPo3bs3NjY2Ro8Kzpw5Q5kyZdLX4KuNPXv2EBERobmFjOTJkyfcvXtXFbF1XkQIwcCBA/n777+NzjFy5coVtm7dStGiRU2szgJIKbN9ATuAc5m8upP6IM9YNjqLOsoY/lYCbgKVSU1WH5ahTFngXE56pJS8/vrrUsM8vPXWW7JixYpSr9fn+d579+7JiRMnygcPHphBmfkZOnSodHV1lc+ePVNaiiox5jtjTdy+fVsKIeRXX31ldB0pKSkmVGR6gFCZyTM1xxGBlLKtlPLVTF5rgftCCB8Aw99MwxBKKe8a/l4H9gD1gCiguBAizbHoC+R/yYrCPHv2jClTprBr1y6lpRhF//79uXHjBocOHcrzvd7e3nz++eeUKFHCDMrMS0JCAqtWreLtt9/G2dlZaTmqJG3PSFJSEikpKTmUtj7Kli1Lq1atWLRoUZ7dW2mfV62LRfLrGloHDDK8HwS8tBBXCOEuhHA0vPcCmgEXDNZpN9Azu/vVhoODA9OnT2fHjh1KSzGKgIAAnJ2d8xwy4MKFC+zdu1eVy2cBNm7cSExMDH379lVaiqq5ePEiPj4+Jg/ZYCkGDBhAWFgYhw8fztN97733Hl26dDGTKvOTX0PwHdBOCHEVaGs4RgjhJ4T4w1CmBhAqhDhN6oP/OyllWlziccBHQogwUucM5uZTj+LY2dlx+fJlvvnmG6WlGIWrqyvdunVjxYoVJCcn5/q+X3/9lS5dupCQkGBGdeZj/vz5lC5dmrZt2yotRdVUqVKFHj16ULZsWaWlGEWPHj1wdnbO86Rx/fr1ady4sZlUWYDM/EXW/lLLHIFafabr16+XgAwJCcn1PU+fPpUHDx40oyrzce/ePWlrayuDgoKUlqJhBfTt21d6eHjIxMREpaWYHIydI9AwjilTptC+fXulZRhFhw4d8Pb2Zt68ebm+x8XFhSZNmphRlfn466+/0Ol0DBo0KOfCGrni3r17nDp1SmkZRjFgwAAePXrEpk2bclU+NDRUtS7RNDRDYCbc3Nzw9vZWZRhmOzs7Bg4cyIYNG4iIiMix/Keffmq2dJfmRkrJ/Pnzady4MdWrV1daToGhc+fODB8+XGkZRtGuXTtKlSrFwoULcyx7+/ZtGjZsyPfff28BZeZDMwRmYsSIESxcuBBHR0elpRjFkCFD0Ol0OU4ax8fHs379ek6fPm0hZaYlNDSU8+fPM2TIEKWlFCh++eUXFixYkHNBK8TOzo5+/fqxYcMGctqzVKpUKZYtW6b6SLVaPgIzc+fOHby9vbG3t1daSp5p2rQpjx8/5vz589mGk5ZSkpSUpEqjN2rUKP7880/u3bun7SbWSOf8+fO8+uqrTJ06VbXhUjJDy0egAIcOHaJ8+fK59jVaG0OHDuXixYscOXIk0+t6vR69Xo8QQpVGICEhgaVLlxIQEKAZATNw+fJlPvroI5KSkpSWkmdq1apFkyZN+OOPP7LcU3D06FFmz56t2pVyGdEMgRlp0KABEydOpH79+kpLMYrAwECKFCmS5aTxrl27qFSpEufPn7ewMtOwdu1aoqOjGTx4sNJSCiTXrl3j999/V63b8L333uPSpUscOHAg0+vBwcGMGzdOlbGVXkRzDWlky+DBg1m9ejUREREvJZo5ePAgU6dOZcmSJTg5OSmk0Hhat27NzZs3CQsLU+2OUGtGp9Px5MkT3N0zjU5v9cTFxeHj40NAQECm8x1SSsLDw1W1Z0JzDSnI4cOHVZu9aejQocTGxmaap6Bp06asXr1alUbg0qVL7NmzR9U5JKwdW1vbdCOgxg6ni4sL/fr1Izg4ONM0lkIIVRmB7NAMgQX4v//7P8aPH6/KH0OLFi2oWbMmv//++3PnT58+TWxsrEKq8s+sWbOwt7fXVguZmZSUFLp06cKXX36ptBSjeO+994iPj2fJkiXp56SUvPXWW8yZM0dBZaZFMwQWYPbs2Rw+fFiVidyFEIwcOZLQ0FCOHj0KpE4S9+jRgx49eiiszjji4+OZP38+b7/9NqVKZRU5XcMU2NnZ4ePjg4eHh9JSjOL111+nbt26zJkzJ70j9+TJExwdHVWdiOclMttubO0vtYSYKCjExMTIokWLykGDBqWfO3LkiGpDSsybN08Ccvfu3UpL0VABM2bMkIBqv+8ZQQsxoSznzp2jTZs2XLt2TWkpeaZYsWIMGDCAZcuWpScdadiwoWpDSsycOZPq1avTsmVLpaUUGqSUqg454ebmxi+//EJ0dLQqE+/khGYILISHhwfh4eHcvn1baSlG8a9//YvExESmTZvGZ599xoMHmaaesHpOnjzJkSNHGDFihCpddWrljz/+oF69epw7d05pKXmmaNGivPvuu6xcuZKJEydSrly5AmcMtOWjFkRKqeqHzxtvvMGlS5eIiYnh2rVr+Pr6Ki0pzwwdOpTly5cTHh6u2mWNaiQqKorVq1fTv3//l5Yhq4Hr16/zyiuvMGzYMGrVqsWHH36otCSj0JaPWgFCCPR6PeHh4UpLMYqRI0cSGRnJggULVGkEIiIiWLx4MYMHD9aMgIXx9PTk/fffV6URAKhUqRJdu3Zl9erVvP/++0rLMTmaIbAwffv2pV27dqoMW9utWzfKlCnD7NmzlZZiFL///jtJSUmMHj1aaSmFlmXLlrF48WKlZRhFqVKliIyMZPny5UpLMTn5cg0JITyA5UAFUpPSB0opo18o0xqYnuFUdaCPlDJECDEfaAnEGK4NllKeyqldtbqGALZu3cqjR48IDAxU1UYmvV5P3bp1KVOmDFu2bOH48eOqCp0RHx9P+fLlady4sWo39xUE2rVrh06nU11O7+vXr1OjRg08PDwoXbo0oaGhqnTzmss1FATslFJWAXYajp9DSrlbSllXSlkXaAM8A7ZlKPLvtOu5MQJqp3379vTt21dVRgBSH6StWrWib9++uLq6Mm3aNKUl5YnFixcTGRnJ2LFjlZZSqFmyZIkq83lXqlSJmzdvMmHCBE6cOMGePXuUlmRS8jsiuAy0klLeE0L4AHuklNWyKT8MaCml7G84ng9skFKuzEu7ah4RACQmJrJo0SIaNWrEa6+9prScPPPJJ5/w008/cf36dcqVK6e0nBzR6/W8+uqrODg4cPLkSVX25AoaycnJ2NnZqeL/Qq/XY2OT2mdOSEigQoUK1K5dm23btuVwp/VhrhFBKSnlPcP7CCCnbZp9gKUvnJsshDgjhJguhMgylrEQYpgQIlQIEZpTsghrJz4+nrFjx2Yav8cauXTp0nPL/kaPHo0Qgp9++kk5UXkgJCSEixcvMm7cOFU8eAo6ly9fpkqVKmzevFlpKbli1KhRDBgwACklTk5OjB07lu3bt3P8+HGlpZmOzHaZZXwBO4Bzmby6A49fKBudTT0+QCRg/8I5ATgCC4D/5KRHFpCdxVevXlVNcvvevXtLDw8PGR8fn35uwIABskiRIvL+/fsKKssZvV4v69WrJ6tUqSJTUlKUlqMhpUxKSpI9evSQ+/fvV1pKrvjqq6/kuHHj0o9jYmKkm5ub7Nmzp4KqjIMsdhbnK9QDcBnwkf97qF/OpuxoYHY211uR6iYqFIYgjeTkZKUl5MjDhw/lrl27njt36dIlaWNjIz/99FOFVOWOjRs3SkDOmzdPaSkaBYjx48dLIYS8dOmS0lLyRFaGIL+uoXXAIMP7QcDabMr25QW3kGFeAZE6XvcndaRRaNizZw/ly5cnLCxMaSnZ4unpSevWrZ87V61aNfr27ctvv/2WY15XpZBSMnHiRMqXL6/6nLIFkbi4OIKDg5WWkSX37t3j0KFDmV4bPXo0Tk5OTJo0ycKqzEN+DcF3QDshxFWgreEYIYSfEOKPtEJCiApAWWDvC/cvFkKcBc4CXkDB+FfNJdWrV6devXpWm8rv+PHjtG/fnlu3bmV6/fPPPyc+Pt5qVxDt2LGDw4cPExQUpMqc0QWdOXPmEBgYyJkzZ5SWkik//vgjb7zxBhERES9dK1myJB988AGLFy9WbYa+58hsmGDtr4LkGrJmVq9eLatVqyZjYmKyLNO3b1/p4uJidXMFOp1O1q9fX5YvX14mJCQoLUcjE549e2bV8wRPnz6VW7ZsyfL6w4cPpaurqwwICLCgqvyBOeYIlHoVNEMQHR0tFyxYoLSMTNHpdNlev3jxorS1tZWjRo2ykKLcsWzZMgnIhQsXKi1FIxdY28KJ3Or56quvJCCPHj1qZkWmIStDoIWYsAJmzZrF4MGDuXr1qtJSgNR107t37079gthk/xWpXr06w4YNY+bMmVy+fNlCCrMnKSmJCRMmULt2bfr166e0HI0cmDVrFs2bN0en0yktBYAzZ87QpEkTrly5kmPZMWPG4OnpqdoMhGlohsAK+OCDDzh+/DhVqlRRWgqQuu6+TZs2uV7n/eWXX1KkSBHGjRtnZmW5Y86cOVy7do3vvvtOdTu4CyMeHh54e3vz5MkTpaUA8ODBA+Lj4ylRokSOZYsVK8bnn3/Ojh072LhxowXUmQctDLWVkZCQoHgy+OTkZJYtW0b//v1zHBGk8e233zJ+/Hj27NmjaMKXhw8fUq1aNerUqcPOnTu1DWQaRiFl7kPGJycnU6dOHZKSkjh37pziv9/s0MJQq4BZs2ZRo0YNRXtGUkrs7e0ZMGBAro0ApA6Ry5Yty4cffkhycrIZFWZPUFAQT5484ddff9WMgMqIiIjgr7/+Uqz9pKQkVq5ciV6vz9N3x97enp9//plr164xffr0nG+wQjRDYEXUq1ePli1bKuYrvXLlCrVr1+bkyZN5vtfZ2ZnffvuNs2fPKrac9ODBg8ydO5ePPvqIWrVqKaJBw3i+//57hg0bpti+lEWLFtGrVy/279+f53vbtWtHQEAAkyZNUme+kcxmkK39VdBWDVkLR48elQ0aNJD//POP0XX06NFDOjk5yatXr5pQWc4kJSXJ2rVry7Jly8rY2FiLtq1hGh49eiSvXLmiWPs6nU5u27bN6Ptv3LghnZ2dZefOna1uFVQaaMtH1cONGzfk6NGjZWJiosXbzu8X+O7du7JYsWKyTZs2Fv0xfPnllxKQq1evtlibGubj7t27FmsrJSVFPnr0yCR1/fTTTxKQ8+fPN0l9pkYzBCoiJCREuri4yCNHjlikvStXrsgpU6aYLCjbzJkzJSB/+uknk9SXE0eOHJG2trbynXfesUh7GuZl/vz50snJSZ47d84i7U2ePFmWKVNG3rt3L9916XQ62bx5c+nm5iZv375tAnWmRTMEKuPBgwcWa+uLL76Q7u7u+XIJZUSv18uuXbtKBwcHeeLECZPUmRUxMTGySpUq0tfXV0ZHR5u1LQ3L8ODBAzlu3Dj59OlTi7QXGhoqx40bZ7IR7NWrV2XRokVl06ZNZVJSkknqNBWaIVApISEhZu9Z6PV6eevWLZPWGRkZKUuXLi2rVKlismH3i+h0OhkQECBtbW3lnj17zNKGhrIkJiaazcVozsi/S5YskYD897//bbY2jEEzBCokKipKFitWTL7//vtmqT8kJMSsvtj9+/dLe3t72a5dO7P86CZNmiQB+eOPP5q8bg3liYqKkn5+fmb5/01MTJStWrWS3333ncnrTmP48OESkIsWLTJbG3lFMwQqJTQ09LmEMKbiyZMn0tPTUw4YMMDkdWdk7ty5EpDDhw83ac9u3rx5EpD9+vWz2hUaGvlDp9PJAQMGyA0bNpi87vj4eDlgwAC5ePFik9edRpqxsbe3fymfh1JohkDlxMfHy+nTp5s0y9bJkyfl48ePTVZfVgQFBUlAjh492iQP7VWrVklbW1vZtm1bLbJoIcIU82Z6vV7GxcWZQE3uePTokaxRo4YsWrSo3Ldvn8XazQrNEKicpUuXSiDfX6bVq1dbfGmbXq+XY8aMSR8Z5GcCbfbs2dLGxkY2btxY2y9QiPj777+li4uL3LhxY77qGT58uGzYsKHFJqKllDI8PFxWq1ZNFilSRG7dutXoeuLi4uSMGTPy1ZnSDEEB4NixY+nvjenV6PV62aFDB9miRYscw0ubGr1enz4yaNOmTZ7zFyQkJMixY8dKQHbs2FEzAoWMmJgYOWLEiHwvPFizZo2cOHGixd2JERERsnbt2tLGxkZ+9913eW4/LCxM1q1bVwoh5KFDh4zWYRZDAPQCzgN6wC+bch1IzW8cBgRlOF8ROGI4vxxwyE27hdUQpHHhwgXp6emZbdKMjJw6dSp9WP3kyRP57Nkzc8rLlgULFkgHBwfp6ekpFyxYkCuDtG/fPlm7dm0JyFGjRlndkjwNy5KSkiJHjx6dq302er1ezp07Vy5btswCyrLnyZMnslevXhKQrVq1kmfOnMnxnsTERPnDDz9IZ2dn6ebmlu8RkbkMQQ2gGrAnK0MA2ALXgEqAA3AaqGm4tgLoY3g/E/hXbtot7Ibg9u3bMjAwUEZEREgpUxPJh4WFPVcmbS4hOjpaOjs7y0GDBllaZpacP39eNmnSRAKyatWq8tdff5W3bt16rpf06NEjuWLFCtmuXTsJSB8fH7l+/XoFVWtYC+fPn5fe3t7pyZzi4+Ofm+vS6/UyMjIy/bhly5aya9euFteZGXq9Xs6aNUt6eHhIGxsb6e/vL9evX//cCFev18vLly/LyZMny9KlS0tAduvWzSTLyM3qGsrBEDQBtmY4/szwEsBDwC6zctm9CrsheJHevXtLX1/f9ON+/frJVq1apR9v2rTJbGv5jUWn08mlS5dKPz8/CUhAuru7y1deeUX6+Pikn/Px8ZHff/+9RSf4NKyf2NjY9NHkH3/8IYH0ncFfffWVdHZ2Tp8HePDggcVdoTnx8OFDOW7cOFmiRAkJSBsbG+nr6ysrV64sXV1d07//bdu2lZs3bzaZKysrQ2CX2+B0+aAMcCfDcTjQCPAEHkspUzKcL5NVJUKIYcAwgHLlyplHqUqZPHnyc9nB3njjDWJjY9OPO3bsqISsbLGxsaFPnz707t2bCxcusGPHDi5dukRMTAxFihShYsWKNG/enObNm2vJZTReomjRounv69evz6RJk3B3dwdSv+8eHh6kpKQ+WnKTYMbSeHp68t133/H111+za9cuDh06xK1bt0hOTsbDw4NXX32Vdu3aUalSJYvoyTExjRBiB+CdyaUJUsq1hjJ7gE+klC9lixFC9AQ6SCnfMxwPINUQfAkcllK+YjhfFtgspXw1J9EFOTGNhoaGhrnIKjFNjiMCKWXbfLZ9Fyib4djXcC4KKC6EsDOMCtLOa2hoaGhYEEskpjkGVBFCVBRCOAB9gHUGf9VuoKeh3CBgrQX0aGhoaGhkIF+GQAgRIIQIJ3Wid6MQYqvhfGkhxCYAQ2//A2ArcBFYIaU8b6hiHPCRECKM1DmDufnRo6GhoaGRd7Tk9RoaGhqFBC15vYaGhoZGpmiGQENDQ6OQoxkCDQ0NjUKOZgg0NDQ0CjmqnCwWQkQCt5TWYQRepIbVKExon7lwoH1mdVBeSvnSVmtVGgK1IoQIzWzGviCjfebCgfaZ1Y3mGtLQ0NAo5GiGQENDQ6OQoxkCyzJbaQEKoH3mwoH2mVWMNkegoaGhUcjRRgQaGhoahRzNEGhoaGgUcjRDoABCiI+FEFII4aW0FnMjhPhBCHFJCHFGCLFGCFFcaU3mQgjRQQhxWQgRJoQIUlqPuRFClBVC7BZCXBBCnBdCjFZak6UQQtgKIU4KITYorcUUaIbAwhgysb0F3FZai4XYDrwqpawNXCE1X3WBQwhhC/wX6AjUBPoKIWoqq8rspAAfSylrAo2BUYXgM6cxmtSw+gUCzRBYnunAp6Qmpy7wSCm3ZchLfZjUTHQFkYZAmJTyupQyCVgGdFdYk1mRUt6TUp4wvI8l9cGYZd7xgoIQwhfoDPyhtBZToRkCCyKE6A7clVKeVlqLQgwFNistwkyUAe5kOA6nEDwU0xBCVADqAUcUlmIJfiK1M6dXWIfJyDFnsUbeEELsALwzuTQBGE+qW6hAkd1nllKuNZSZQKorYbEltWmYHyFEUWAVMEZK+URpPeZECNEFeCClPC6EaKWwHJOhGQITI6Vsm9l5IcRrQEXgtBACUl0kJ4QQDaWUERaUaHKy+sxpCCEGA12AN2XB3bhyFyib4djXcK5AI4SwJ9UILJZSrlZajwVoBnQTQnQCnIBiQoi/pJTvKKwrX2gbyhRCCHET8JNSqi16YZ4QQnQAfgRaSikjldZjLoQQdqROhr9JqgE4BvTLkJ+7wCFSezQLgEdSyjEKy7E4hhHBJ1LKLgpLyTfaHIGGufkNcAW2CyFOCSFmKi3IHBgmxD8AtpI6abqiIBsBA82AAUAbw//tKUNPWUNlaCMCDQ0NjUKONiLQ0NDQKORohkBDQ0OjkKMZAg0NDY1CjmYINDQ0NAo5miHQ0NDQKORohkBDQ0OjkKMZAg0NDY1Czv8HajiyCNwUzVoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 参考案例\n",
    "import numpy as np\n",
    "import torch\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Make some fake data.\n",
    "x = np.arange(-5, 5, 0.02)\n",
    "f = np.sin(x)\n",
    "df = []\n",
    "for i in x:\n",
    "# 对于x中的每个值，都求一下导\n",
    "    v = torch.tensor(i, requires_grad=True)\n",
    "    y = torch.sin(v)\n",
    "    y.backward()\n",
    "    df.append(v.grad)\n",
    "\n",
    "# 使用预定义的标签创建绘图\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(x, f, 'k', label='f(x)')\n",
    "ax.plot(x, df, 'k:', label='df(x)')\n",
    "\n",
    "legend = ax.legend(loc='upper left', shadow=True, fontsize='x-large')\n",
    "\n",
    "# Put a nicer background color on the legend.\n",
    "legend.get_frame().set_facecolor('C0')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "1d863003",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/woyouhenduoyiwen/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/ipykernel_launcher.py:11: DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`\n",
      "  # This is added back by InteractiveShellApp.init_path()\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg height=\"184.15625pt\" version=\"1.1\" viewBox=\"0 0 258.160937 184.15625\" width=\"258.160937pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2021-07-05T02:00:12.209556</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.4.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 184.15625 \n",
       "L 258.160937 184.15625 \n",
       "L 258.160937 -0 \n",
       "L 0 -0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 52.160938 146.6 \n",
       "L 247.460938 146.6 \n",
       "L 247.460938 10.7 \n",
       "L 52.160938 10.7 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 61.03821 146.6 \n",
       "L 61.03821 10.7 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"md015c778e2\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"61.03821\" xlink:href=\"#md015c778e2\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -3.5 \n",
       "\" id=\"m5e1d2c6578\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"61.03821\" xlink:href=\"#m5e1d2c6578\" y=\"10.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(57.85696 161.198437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 122.260781 146.6 \n",
       "L 122.260781 10.7 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"122.260781\" xlink:href=\"#md015c778e2\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"122.260781\" xlink:href=\"#m5e1d2c6578\" y=\"10.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(119.079531 161.198437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 183.483351 146.6 \n",
       "L 183.483351 10.7 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"183.483351\" xlink:href=\"#md015c778e2\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"183.483351\" xlink:href=\"#m5e1d2c6578\" y=\"10.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(180.302101 161.198437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 244.705922 146.6 \n",
       "L 244.705922 10.7 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"244.705922\" xlink:href=\"#md015c778e2\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"244.705922\" xlink:href=\"#m5e1d2c6578\" y=\"10.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 3 -->\n",
       "      <g transform=\"translate(241.524672 161.198437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" id=\"DejaVuSans-33\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_5\">\n",
       "     <!-- x -->\n",
       "     <g transform=\"translate(146.851563 174.876562)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" id=\"DejaVuSans-78\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 52.160938 142.243155 \n",
       "L 247.460938 142.243155 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m519f9b9ab3\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#m519f9b9ab3\" y=\"142.243155\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 3.5 0 \n",
       "\" id=\"mb2110e9850\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.460938\" xlink:href=\"#mb2110e9850\" y=\"142.243155\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- −1.0 -->\n",
       "      <g transform=\"translate(20.878125 146.042374)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" id=\"DejaVuSans-2212\" transform=\"scale(0.015625)\"/>\n",
       "        <path d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" id=\"DejaVuSans-2e\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 52.160938 110.901684 \n",
       "L 247.460938 110.901684 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#m519f9b9ab3\" y=\"110.901684\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.460938\" xlink:href=\"#mb2110e9850\" y=\"110.901684\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- −0.5 -->\n",
       "      <g transform=\"translate(20.878125 114.700903)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-35\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 52.160938 79.560214 \n",
       "L 247.460938 79.560214 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#m519f9b9ab3\" y=\"79.560214\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.460938\" xlink:href=\"#mb2110e9850\" y=\"79.560214\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(29.257812 83.359433)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 52.160938 48.218743 \n",
       "L 247.460938 48.218743 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#m519f9b9ab3\" y=\"48.218743\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.460938\" xlink:href=\"#mb2110e9850\" y=\"48.218743\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(29.257812 52.017962)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_25\">\n",
       "      <path clip-path=\"url(#p162488d2f3)\" d=\"M 52.160938 16.877273 \n",
       "L 247.460938 16.877273 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_26\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#m519f9b9ab3\" y=\"16.877273\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.460938\" xlink:href=\"#mb2110e9850\" y=\"16.877273\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(29.257812 20.676491)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- f(x) -->\n",
       "     <g transform=\"translate(14.798438 87.271094)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-66\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1984 4856 \n",
       "Q 1566 4138 1362 3434 \n",
       "Q 1159 2731 1159 2009 \n",
       "Q 1159 1288 1364 580 \n",
       "Q 1569 -128 1984 -844 \n",
       "L 1484 -844 \n",
       "Q 1016 -109 783 600 \n",
       "Q 550 1309 550 2009 \n",
       "Q 550 2706 781 3412 \n",
       "Q 1013 4119 1484 4856 \n",
       "L 1984 4856 \n",
       "z\n",
       "\" id=\"DejaVuSans-28\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 513 4856 \n",
       "L 1013 4856 \n",
       "Q 1481 4119 1714 3412 \n",
       "Q 1947 2706 1947 2009 \n",
       "Q 1947 1309 1714 600 \n",
       "Q 1481 -109 1013 -844 \n",
       "L 513 -844 \n",
       "Q 928 -128 1133 580 \n",
       "Q 1338 1288 1338 2009 \n",
       "Q 1338 2731 1133 3434 \n",
       "Q 928 4138 513 4856 \n",
       "z\n",
       "\" id=\"DejaVuSans-29\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-28\"/>\n",
       "      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-78\"/>\n",
       "      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-29\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_28\">\n",
       "    <path clip-path=\"url(#p162488d2f3)\" d=\"M 61.03821 79.560214 \n",
       "L 67.160467 73.302362 \n",
       "L 73.282724 67.107036 \n",
       "L 79.404981 61.036138 \n",
       "L 85.527238 55.150327 \n",
       "L 91.649495 49.508411 \n",
       "L 97.771753 44.166763 \n",
       "L 103.89401 39.178754 \n",
       "L 110.016267 34.594224 \n",
       "L 116.138524 30.458979 \n",
       "L 122.260781 26.814338 \n",
       "L 128.383038 23.696715 \n",
       "L 134.505295 21.137263 \n",
       "L 140.627552 19.161553 \n",
       "L 146.749809 17.789326 \n",
       "L 152.872066 17.034294 \n",
       "L 158.994323 16.904001 \n",
       "L 165.11658 17.399747 \n",
       "L 171.238837 18.51658 \n",
       "L 177.361094 20.243341 \n",
       "L 183.483351 22.562777 \n",
       "L 189.605608 25.451712 \n",
       "L 195.727865 28.881281 \n",
       "L 201.850122 32.817218 \n",
       "L 207.97238 37.220195 \n",
       "L 214.094637 42.04622 \n",
       "L 220.216894 47.247072 \n",
       "L 226.339151 52.770786 \n",
       "L 232.461408 58.562171 \n",
       "L 238.583665 64.563362 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_29\">\n",
       "    <path clip-path=\"url(#p162488d2f3)\" d=\"M 61.03821 16.877273 \n",
       "L 67.160467 17.190426 \n",
       "L 73.282724 18.126758 \n",
       "L 79.404981 19.676913 \n",
       "L 85.527238 21.825402 \n",
       "L 91.649495 24.550758 \n",
       "L 97.771753 27.82575 \n",
       "L 103.89401 31.617656 \n",
       "L 110.016267 35.888588 \n",
       "L 116.138524 40.595873 \n",
       "L 122.260781 45.692476 \n",
       "L 128.383038 51.127475 \n",
       "L 134.505295 56.846564 \n",
       "L 140.627552 62.7926 \n",
       "L 146.749809 68.906173 \n",
       "L 152.872066 75.126198 \n",
       "L 158.994323 81.390526 \n",
       "L 165.11658 87.636566 \n",
       "L 171.238837 93.801909 \n",
       "L 177.361094 99.824955 \n",
       "L 183.483351 105.645521 \n",
       "L 189.605608 111.205452 \n",
       "L 195.727865 116.449195 \n",
       "L 201.850122 121.324354 \n",
       "L 207.97238 125.782221 \n",
       "L 214.094637 129.778252 \n",
       "L 220.216894 133.272521 \n",
       "L 226.339151 136.230115 \n",
       "L 232.461408 138.621481 \n",
       "L 238.583665 140.422727 \n",
       "\" style=\"fill:none;stroke:#bf00bf;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 52.160938 146.6 \n",
       "L 52.160938 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 247.460938 146.6 \n",
       "L 247.460938 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 52.160938 146.6 \n",
       "L 247.460938 146.6 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 52.160938 10.7 \n",
       "L 247.460938 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 59.160938 141.6 \n",
       "L 183.710938 141.6 \n",
       "Q 185.710938 141.6 185.710938 139.6 \n",
       "L 185.710938 111.24375 \n",
       "Q 185.710938 109.24375 183.710938 109.24375 \n",
       "L 59.160938 109.24375 \n",
       "Q 57.160938 109.24375 57.160938 111.24375 \n",
       "L 57.160938 139.6 \n",
       "Q 57.160938 141.6 59.160938 141.6 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_30\">\n",
       "     <path d=\"M 61.160938 117.342187 \n",
       "L 81.160938 117.342187 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_31\"/>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- f(x) -->\n",
       "     <g transform=\"translate(89.160938 120.842187)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-28\"/>\n",
       "      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-78\"/>\n",
       "      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-29\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_32\">\n",
       "     <path d=\"M 61.160938 132.020312 \n",
       "L 81.160938 132.020312 \n",
       "\" style=\"fill:none;stroke:#bf00bf;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_33\"/>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- Tangent line (x=1) -->\n",
       "     <g transform=\"translate(89.160938 135.520312)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M -19 4666 \n",
       "L 3928 4666 \n",
       "L 3928 4134 \n",
       "L 2272 4134 \n",
       "L 2272 0 \n",
       "L 1638 0 \n",
       "L 1638 4134 \n",
       "L -19 4134 \n",
       "L -19 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-54\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2906 1791 \n",
       "Q 2906 2416 2648 2759 \n",
       "Q 2391 3103 1925 3103 \n",
       "Q 1463 3103 1205 2759 \n",
       "Q 947 2416 947 1791 \n",
       "Q 947 1169 1205 825 \n",
       "Q 1463 481 1925 481 \n",
       "Q 2391 481 2648 825 \n",
       "Q 2906 1169 2906 1791 \n",
       "z\n",
       "M 3481 434 \n",
       "Q 3481 -459 3084 -895 \n",
       "Q 2688 -1331 1869 -1331 \n",
       "Q 1566 -1331 1297 -1286 \n",
       "Q 1028 -1241 775 -1147 \n",
       "L 775 -588 \n",
       "Q 1028 -725 1275 -790 \n",
       "Q 1522 -856 1778 -856 \n",
       "Q 2344 -856 2625 -561 \n",
       "Q 2906 -266 2906 331 \n",
       "L 2906 616 \n",
       "Q 2728 306 2450 153 \n",
       "Q 2172 0 1784 0 \n",
       "Q 1141 0 747 490 \n",
       "Q 353 981 353 1791 \n",
       "Q 353 2603 747 3093 \n",
       "Q 1141 3584 1784 3584 \n",
       "Q 2172 3584 2450 3431 \n",
       "Q 2728 3278 2906 2969 \n",
       "L 2906 3500 \n",
       "L 3481 3500 \n",
       "L 3481 434 \n",
       "z\n",
       "\" id=\"DejaVuSans-67\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" id=\"DejaVuSans-65\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 678 2906 \n",
       "L 4684 2906 \n",
       "L 4684 2381 \n",
       "L 678 2381 \n",
       "L 678 2906 \n",
       "z\n",
       "M 678 1631 \n",
       "L 4684 1631 \n",
       "L 4684 1100 \n",
       "L 678 1100 \n",
       "L 678 1631 \n",
       "z\n",
       "\" id=\"DejaVuSans-3d\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-54\"/>\n",
       "      <use x=\"44.583984\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"105.863281\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"169.242188\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"232.71875\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"294.242188\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"357.621094\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"396.830078\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"428.617188\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"456.400391\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"484.183594\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"547.5625\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"609.085938\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"640.873047\" xlink:href=\"#DejaVuSans-28\"/>\n",
       "      <use x=\"679.886719\" xlink:href=\"#DejaVuSans-78\"/>\n",
       "      <use x=\"739.066406\" xlink:href=\"#DejaVuSans-3d\"/>\n",
       "      <use x=\"822.855469\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "      <use x=\"886.478516\" xlink:href=\"#DejaVuSans-29\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p162488d2f3\">\n",
       "   <rect height=\"135.9\" width=\"195.3\" x=\"52.160938\" y=\"10.7\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 存在错误，待定\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "from IPython import display\n",
    "from d2l import torch as d2l\n",
    "\n",
    "x = np.arange(-6, 6, 0.02)\n",
    "def f(x):\n",
    "    return np.sin(x)\n",
    "df = []\n",
    "for i in x:\n",
    "# 对于x中的每个值，都求一下导\n",
    "    v = torch.tensor(i, requires_grad=True)\n",
    "    y = torch.sin(v)\n",
    "    y.backward()\n",
    "    df.append(v.grad)\n",
    "\n",
    "def use_svg_display():  #@save\n",
    "    \"\"\"使用svg格式在Jupyter中显示绘图。\"\"\"\n",
    "    display.set_matplotlib_formats('svg')\n",
    "def set_figsize(figsize=(3.5, 2.5)):  #@save\n",
    "    \"\"\"设置matplotlib的图表大小。\"\"\"\n",
    "    use_svg_display()\n",
    "    d2l.plt.rcParams['figure.figsize'] = figsize\n",
    "#@save\n",
    "def set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend):\n",
    "    \"\"\"设置matplotlib的轴。\"\"\"\n",
    "    axes.set_xlabel(xlabel)\n",
    "    axes.set_ylabel(ylabel)\n",
    "    axes.set_xscale(xscale)\n",
    "    axes.set_yscale(yscale)\n",
    "    axes.set_xlim(xlim)\n",
    "    axes.set_ylim(ylim)\n",
    "    if legend:\n",
    "        axes.legend(legend)\n",
    "    axes.grid()\n",
    "#@save\n",
    "def plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None,\n",
    "         ylim=None, xscale='linear', yscale='linear',\n",
    "         fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None):\n",
    "    \"\"\"绘制数据点。\"\"\"\n",
    "    if legend is None:\n",
    "        legend = []\n",
    "\n",
    "    set_figsize(figsize)\n",
    "    axes = axes if axes else d2l.plt.gca()\n",
    "\n",
    "    # 如果 `X` 有一个轴，输出True\n",
    "    def has_one_axis(X):\n",
    "        return (hasattr(X, \"ndim\") and X.ndim == 1 or\n",
    "                isinstance(X, list) and not hasattr(X[0], \"__len__\"))\n",
    "\n",
    "    if has_one_axis(X):\n",
    "        X = [X]\n",
    "    if Y is None:\n",
    "        X, Y = [[]] * len(X), X\n",
    "    elif has_one_axis(Y):\n",
    "        Y = [Y]\n",
    "    if len(X) != len(Y):\n",
    "        X = X * len(Y)\n",
    "    axes.cla()\n",
    "    for x, y, fmt in zip(X, Y, fmts):\n",
    "        if len(x):\n",
    "            axes.plot(x, y, fmt)\n",
    "        else:\n",
    "            axes.plot(y, fmt)\n",
    "    set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)\n",
    "x = np.arange(0, 3, 0.1)\n",
    "plot(x, [f(x), np.cos(x)], 'x', 'f(x)', legend=['f(x)', 'Tangent line (x=1)'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pytorch",
   "language": "python",
   "name": "pytorch"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
